The visual appearance of an object or a scene can be influenced by several factors. One of these factors is the characteristic of the light that illuminates the scene. For example, a scene at the beach on a bright day appears different from the same scene when viewed on an overcast day, a foggy day, or on a moonlit night. A lifeguard stand at the same beach would appear different when illuminated by a flashlight on a dark night or when illuminated by a halogen spotlight on a police cruiser. In each case, the type of light source that illuminates the scene is different, and so the appearance of the scene is different. One difference in the type of light in each of these cases is the spectral content of the light. In the visual spectrum, the spectral content of light often is described by its color. Thus, the yellow light of the sun, the blue light of a halogen lamp, and other subtle differences in the spectral content of the light that illuminates the beach all lead to a distinctively different visual experience.
A more precise way of characterizing light is to describe it in terms of its wavelength. Thus, white light (theoretically, at least) contains all wavelengths in the visible spectrum. Red light occupies a longer-wavelength end of the visible spectrum. Violet light occupies a shorter-wavelength end of the visible spectrum. Even electromagnetic radiation that is not visible can be characterized in terms of its wavelength. For example, infrared light, which is invisible to the naked eye, has a wavelength longer than that of visible red light. Ultraviolet, which also is invisible to the naked eye, has a wavelength shorter than that of visible violet light.
A second factor that influences the appearance of a scene relates to how light is reflected from objects in the scene. An object that reflects no light appears to be black. Alternatively, it could be said that a black object is one that absorbs all wavelengths of light. An object that reflects red light appears to the naked eye to be red, an object that reflects green light appears to the naked eye to be green, and so on.
A third factor that can influence the appearance of a scene may take the form of optical filters that selectively transmit different wavelengths of light. For example, a person viewing the beach while wearing sunglasses experiences a different view from that of another person who views the beach through the naked eye. Some types of sunglasses operate by selectively transmitting light only in one part of the visual spectrum. That is, they tend to block light in other parts of the visual spectrum. Sometimes this filtering can cause a scene to be viewed more clearly than would be the case when viewing the scene with the naked eye. A person viewing the beach through yellow sunglasses on a foggy day may see the beach more clearly than someone viewing the same beach with the naked eye. Yellow sunglasses selectively transmit light at the red-orange-yellow end of the visible spectrum (i.e. they attenuate light received at the blue-indigo-violet end of the visible spectrum) and thus can materially aid in viewing objects through fog. This aid arises because fog tends to scatter light in the blue part of the visible spectrum. Yellow sunglasses, by not passing the scattered light, can provide a relatively clearer view of a foggy scene. At the same time, a person wearing yellow sunglasses may fail to notice a blue object, such as a blue beach umbrella in the same scene because the light that reaches the person's eyes is somewhat devoid of information at the blue end of the spectrum.
Yet another factor that influences the appearance of a scene relates to a detector that receives the light and converts the received light into an image that can be viewed with the naked eye. Infrared film is one example of such a detector. Some types of infrared film display a high-temperature object in an image as white and a low-temperature image as black. As a basic illustration, the human eye functions as a detector when viewing a scene. As such, different persons may have different ability to detect visible light in different bands (i.e. colors). For example, a colorblind individual may not be able to discriminate between red and green. This individual may have great difficulty in seeing a red bird sitting on a branch of a green tree.
These simple examples illustrate that the appearance of a scene or an object depends upon several factors. One of these factors is the spectral content of the light source that illuminates a scene. A second factor is the manner in which objects reflect light that illuminates them. A third factor is the effect of any filters that selectively transmit light received from a scene according to the wavelength of the light. Yet a fourth factor relates to the characteristics of a detector that receives the light.
These observations about the interaction of light with objects lead to many practical applications that are important from an economic point of view. One example occurs in systems that help a pilot to visualize a runway. For example, if mobile machinery blocks a runway at night, such machinery may be invisible to the naked eye. Because there may not be enough illumination arriving at the surface of the machinery, the machinery would be invisible to the naked eye and probably would not appear in an image generated by a standard camera whose detector (e.g. conventional photographic film) is sensitive to visible wavelengths of light. However, recognizing that the heat generated by an engine radiates very strongly in the range of infrared wavelengths, a camera loaded with infrared film is likely to see the machinery very clearly. Likewise, a television camera with a detector that was sensitive to radiation in the infrared range could produce a signal that could be displayed on a monitor. A pilot then may view the detected image on a monitor in order more safely land an airplane. In another example that relates to the hazy day at the beach, haze can obscure daytime images of a runway by scattering the shorter wavelengths at the blue end of the visible spectrum. Images using only yellow, red, and near-infrared wavelengths, however, can show the runway more clearly through haze.
One way to take advantage of the information available in the different wavelength ranges just mentioned is to employ a technique called multi-spectral imaging. Multi-spectral imaging can be used to view a scene from more than one perspective, where each perspective is that of the image viewed in a different radiation band. Hence, one perspective may be the scene as viewed in the visible radiation band (e.g., the scene may be viewed in the visual and the near infrared spectral bands). Another perspective may be the same image viewed in a different spectral band (e.g., in the long infrared band). In order to view a scene from these different perspectives simultaneously, light from the scene must be directed to different detectors, each sensitive to a particular range of wavelengths. A range of wavelengths is sometimes referred to as a spectral band or, if the context is clear, simply a band.
Multiple images of the same scene using two or more wavelength ranges may be combined to reveal more information than any one image could provide. Such multi-spectral imaging may be effectively applied to military target acquisition and detection, aircraft runway visibility enhancement, aircraft runway obstacle detection, and the like.
The use of multi-spectral imaging is not restricted to military or avionics use. For example, medical applications include imaging systems for surgical support. Also, multi-spectral imaging is used in maritime applications and in earth-observing satellite systems. Such multi-spectral imaging is also used in astronomy where, for example, multi-spectral imaging systems are used to probe the far reaches of space.
Because separate perspectives of a single scene are typically combined into a composite image, good spatial registration amongst the different perspectives is also required. Good spatial registration occurs when the features of every object in the scene appear in the same location in each of the images from multiple different imagers. When two or more images of different wavelength ranges are superimposed, exact spatial registration means that details observable in different wavelength ranges will be shown with the correct spatial relationship to one another within the scene.
Light in substantially different wavelength ranges requires a different kind of light sensor or detector for each wavelength range. A typical silicon-based charge-coupled device (CCD) detector, like those used in a common video camera, for example, is sensitive to wavelengths of light that range from about 0.4 to about 1.0 microns. This range of wavelengths (i.e. this band) includes visible light (VIS) with wavelengths from about 0.4 to 0.7 microns and near infrared (NIR) with wavelengths from about 0.7 to 1.0 microns. The most useful infrared wavelength ranges are included in three bands that are not significantly absorbed by air. One of these bands is the short infrared (SIR) band with wavelengths from about 1 to 3 microns. Another of these useful bands includes the mid wave infrared (MWIR) band with wavelengths from about 3 to 5 microns. Yet another useful infrared band is the long wave infrared (LWIR) band with wavelengths from about 8 to 14 microns. Each of these bands requires a detector that is different from a typical silicon CCD detector. The detectors further are different from each other.
One problem with known multi-spectral imaging systems is that the optics involved in capturing an image from a scene may not be capable of processing radiation in more than one range of wavelength. Hence, traditional multi-spectral imaging systems use separate optical paths to process images for different detectors. By using separate optical paths to process an image at different wavelength, there is an inherent misregistration error. This misregistration error is an artifact of the fact that the scene processed by two distinct optical paths can never be exactly the same.
Several approaches are available for reducing the effect of misregistration error. One choice is to accept different views of the scene on the two detectors and to superimpose the images without any attempt to mitigate the error. This choice results in a distorted image that may obscure important details. Another choice is to augment the multi-spectral imaging system 5 by adding mechanisms to estimate the distance to a scene and to adjust the direction in which each optical path is pointed according to the relative angle with which each optical path views the scene. This choice has the effect of increasing the cost, size, and weight of a multi-spectral imaging system. The additional mechanisms also require special maintenance in order to derive the benefit offered by the mechanisms. Yet another choice is to convert each detected image to digital form and to use a digital signal processing computer to scale each image and then digitally align multiple images with each other. This technique can also reduce the misregistration error. Unfortunately, algorithms for performing such scaling are extremely complicated and employ heuristic techniques that are not guaranteed to be appropriate in all cases. Additionally, such digitally-based systems are very expensive to develop and test.